Automated Scoring for Reading Comprehension via In-context BERT Tuning

Automated scoring of open-ended student responses has the potential to significantly reduce human grader effort. Recent advances in automated leverage textual representations from pre-trained language models like BERT. Existing approaches train a separate model for each item/question, suitable scenarios essay where items can be different one another. However, these have two limitations: 1) they fail item linkage such as reading comprehension multiple may share passage; 2) are not scalable since storing per is difficult with large models. We report our (grand prize-winning) solution National Assessment Education Progress (NAEP) challenge comprehension. Our approach, in-context BERT fine-tuning, produces single shared all carefully designed input structure provide contextual information on item. experiments demonstrate effectiveness approach which outperforms existing methods. also perform qualitative analysis and discuss limitations approach. (Full version paper found at: https://arxiv.org/abs/2205.09864 implementation https://github.com/ni9elf/automated-scoring )